Comparative effectiveness research (CER) is aiming at informing health care decisions concerning the benefits and risks of different diagnosis and treatment options. The growing number of assessment instruments and treatment options for a given condition, as well as the rapid escalation in their costs, has generated the increasing need for scientifically rigorous comparisons of multiple diagnostic tests and multiple treatments in clinical practice via multivariate meta-analysis. While multivariate meta-analysis methods are certainly very useful as they can provide estimates with better statistical properties, there are many additional challenges compared to traditional univariate meta-analysis. In response to PA-13-303, the overall goal of this proposal is to develop cutting-edge and robust multivariate meta-analysis methods to enhance the consistency, applicability, and generalizability, and a completely open-source, cross-platform, publicly available and easy-to-use R software package. In addition, we will integrate the to-be-developed R package into the publicly available software Open Meta-Analyst for advanced meta-analysis. In this proposal, we will: (1) develop multivariate network meta-analysis frameworks and software to simultaneously compare multiple treatments with multivariate outcomes, and multiple diagnostic tests (which intrinsically involve multivariate outcomes due to the joint interests in prevalence, sensitivity ad specificity); and (2) develop a multivariate visualization tool and nonparametric/parametric methods to detect and adjust for publication bias. We will evaluate the strengths and weaknesses of these proposed methods versus existing meta-analysis methods through many real data analyses and carefully designed simulation studies. The proposed statistical methodology will be broadly applicable to multivariate meta-analysis. Completion of these two aims will directly benefit the CER program by providing state-of-the art methods implemented in user-friendly software using the JAGS and R statistical languages that will be made freely available to the public. It will improve public health by facilitating the diagnosis and treatment of various cancers, cardiovascular, infectious and other diseases.